Secure classifying of data with Gaussian distributions

ABSTRACT

A first party has a data vector x and a second party has a classifier defined as a set of multivariate Gaussian distributions. A secure inner dot product procedure is applied to each multivariate Gaussian distribution and the data vector x to produce a vector a i  for the first party and a vector b i  for the second party for each application. The secure inner dot product is then applied to each vector b i  and the data vector x to produce a scalar r i  for the first party and a scalar q i  for the second party for each application. A summed vector of elements [(a 1 x+q 1 +r 1 ), . . . , (a N x+q N +r N )] is formed, and an index I of the summed vector for a particular element having a maximum value is the class of the data vector x.

FIELD OF THE INVENTION

This invention relates generally to classifying data, and more particularly to securely classifying data.

BACKGROUND OF THE INVENTION

With the availability of global communication networks, it is now common to ‘outsource’ data processing tasks to third parties for a number of reasons. For example, the processing can be done at a reduced cost, or the third party has better computational resources and/or better technologies. One concern with outsourcing data processing is an inappropriate use of confidential information by third parties.

A user or client, perhaps with limited data processing resources, desires to classify data. A third party or server has some of the necessary resources. It is desired to have the third party classify the data without revealing the data to the third party. At the same time, the third party wants to keep its classifier secret. For such applications, conventional cryptography protects only the data during transport, and not the processing by the third party.

In a specific example application, a client, Alice, has a data vector. A server, Bob, has a classifier that uses Gaussian distributions. Bob's classifier can be used to classify Alice's data vector. Alice and Bob would like to engage in a protocol that lets Bob classify Alice's data without revealing the data vector to Bob and the classifier to Alice.

Secure multi-party computations are often analyzed for correctness, security, and overhead. Correctness measures how close a secure process approaches an ideal solution. Security measures the amount of information that can be gained from the multi-party exchange. Overhead is a measure of complexity and efficiency.

A two-party protocol between Alice and Bob is secure when privacy and correctness are guaranteed for both Alice and Bob. The protocol protects privacy when the information that is ‘leaked’ by the distributed computation is limited to the information that can be learned from the designated output of the computation. In a semi-honest case, both parties follow the protocol as prescribed but may record all messages and subsequently deduce information not derivable solely from the protocol output. In the malicious case, however, no assumption is made about the behavior of the parties. It is required that the privacy of one party is preserved even in the case of an arbitrary behavior of another party. A protocol in the semi-honest case can be made secure in the malicious case when accompanied with zero-knowledge proofs that both parties follow the protocol.

Zero Knowledge Protocols

Zero-knowledge or secure multi-party protocols were first described by Yao, “How to generate and exchange secrets,” Proceedings of the 27th IEEE Symposium on Foundations of Computer Science, pp. 162-167, 1986, for a specific problem. Later, that zero-knowledge technique was extended to other problems, Goldreich et al., “How to play any mental game—a completeness theorem for protocols with honest majority,” 19th ACM Symposium on the Theory of Computing, pp 218-229, 1987. However, those theoretical constructs are still too demanding to be of any use for practical applications.

Since then, many secure methods have been described, Chang et al., “Oblivious Polynomial Evaluation and Oblivious Neural Learning,” Advances in Cryptology, Asiacrypt '01, Lecture Notes in Computer Science Vol. 2248, pages 369-384, 2001; Clifton et al., “Tools for Privacy Preserving Distributed Data Mining,” SIGKDD Explorations, 4(2):28-34, 2002; Koller et al., “Protected Interactive 3D Graphics Via Remote Rendering,” SIGGRAPH 2004; Lindell et al., “Privacy preserving data mining,” Advances in Cryptology—Crypto 2000, LNCS 1880, 2000; Naor et al., “Oblivious Polynomial Evaluation,” Proc. of the 31st Symp. on Theory of Computer Science (STOC), pp. 245-254, May 1999; and Du et al., “Privacy-preserving cooperative scientific computations,” 4th IEEE Computer Security Foundations Workshop, pp.273-282, Jun. 11, 2001. A full treatment of the problem can be found in the reference text book by Goldreich, Foundations of Cryptography, Cambridge University Press, 1998, all incorporated herein by reference.

Secure Inner Dot Products

A large number of computer implemented methods and applications require that one computes an inner product. Therefore, protocols and procedures for determining a secure inner dot product (SIP) have been developed. It is understood that these protocols are known to those of ordinary skill in the art.

The protocols can be categorized broadly as cryptographic protocols and algebraic protocols. The protocols provide different levels of security and efficiency. Because computational costs, in general, are constantly decreasing, the main concern in evaluating the protocols is security and communication costs.

Generally, an inner dot product of two vectors x and y produces two scalar values a and b according to x^(t)y=a+b, where t is a transpose operator.

Cryptographic Protocols

Several methods are known for providing secure inner dot products, see Goethals et al., “On private scalar product computation for privacy-preserving data mining,” C. Park and S. Chee, editors, Intl. Conference on Information Security and Cryptology, Vol. 2506 of Lecture Notes in Computer Science, pp. 104-120, 2004; Feigenbaum et al., “Secure multiparty computation of approximations,” Proceedings of the Intl. Colloquium on Automata, Languages and Programming, Vol. 2076 of Lecture Notes in Computer Science, pp. 927-938, 2001; and Canetti et al., “Selective private function evaluation with applications to private statistics,” Proceedings of the ACM symposium on principles of distributed computing, pp. 293-304, 2001, all incorporated herein by reference.

The Goethal et al. protocol has as input private vectors x and y belonging to Bob and Alice respectively, and as output, the scalar values a and b such that a+b=x^(t)y. During an initialization phase, Bob generates a private and public key pair (sk, pk). Bob sends pk to Alice. Then, for each i∈{1, . . . , d}, Bob generates a new random string r_(i), and sends c_(i)=En(pk; x_(i), r_(i)) to Alice. In response, Alice sets $\left. z\leftarrow{\prod\limits_{i = 1}^{d}c_{i}^{y_{i}}} \right.,$ generates a random plaintext b and a random nonce r′, and sends z′=z·En(pk; −b, r′) to Bob. Then, Bob determines a=De(sk; z′)=x^(t)y−b. Goethals et al. describe a proof that the protocol is correct and secure. It can also be shown that the communication overhead is k′/τ, where k′ is the bit size of each encrypted message sent by Alice, and τ=└√{square root over (m/d)}┘ for a large m. In a homomorphic cryptosystem, typical values would be k′≈2048 and m≈2¹⁰²⁴, see Paillier, “Public-key cryptosystems based on composite degree residuosity classes,” J. Stern, editor, Proceedings of Advances in Cryptology—EUROCRYPT '99, Vol. 1592 of Lecture Notes in Computer Science, pp.223-238, 1999; and Damgard et al., “A generalization, simplification and some applications of Paillier's probabilistic public-key system,” Proceedings of the Intl. Workshop on Practice and Theory in Public Key Cryptography, Vol. 1992 of Lecture Notes in Computer Science, pp. 119-136, 2001, all incorporated herein by reference.

Another protocol uses the technique of oblivious polynomial evaluation (OPE), see Naor et al., “Oblivious transfer and polynomial evaluation,” Proceedings of the thirty-first annual ACM symposium on Theory of computing, pp. 245-254, 1999, incorporated herein by reference. The protocol uses an additive oracle which computes every term x_(j)y_(j) of the dot product x^(t)y and expresses the j^(th) term as S_(A) _(j) +S_(B) _(j) . Alice and Bob receive S_(A) _(j) and S_(B) _(j) , respectively for all j. The result x^(t)y is given by ${\sum\limits_{i = 1}^{d}S_{A_{i}}} + {\sum\limits_{i = 1}^{d}{S_{B_{i}}.}}$ Alice and Bob can implement the additive oracle using the OPE.

Canetti et al. describe a protocol which uses homomorphic encryption for determining a weighted sum of a set of numbers from a database. Feigenbaum et al. describe a protocol to determine the L² distance between two vectors, see Feigenbaum et al., “Secure multiparty computation of approximations,” Proceedings of the Intl. Colloquium on Automata, Languages and Programming, Vol.2076 of Lecture Notes in Computer Science, pages 927-938, 2001, all incorporated herein by reference. Both of those protocols can be used to determine dot-products securely.

Algebraic Protocols

Algebraic protocols can also be used for determining secure dot products. However, most of these protocols leak some information, see Du et al., “Privacy-preserving cooperative statistical analysis,” Proceedings of the 17th Annual Computer Security Applications Conference, December 2001; Du et al., “A practical approach to solve secure multi-party computation problems,” Proceedings of New Security Paradigms Workshop, Sep. 23-26 2002; Ioannidis et al., “A secure protocol for computing dot-products in clustered and distributed environments,” Proceedings of the Intl. Conf. on Parallel Processing, 2002; Ravikumar et al., “A secure protocol for computing string distance metrics,” Proceedings of the Workshop on Privacy and Security Aspects of Data Mining, pages 40-46, Brighton, UK, 2004; and Vaidya et al., “Privacy preserving association rule mining in vertically partitioned data,” Proceedings of the Intl. Conf. on Knowledge Discovery and Data Mining. The properties and weaknesses of some of these protocols have been analyzed, see Kiltz et al., “Secure computation of the mean and related statistics,” Proceedings of the Theory of Cryptography Conference, Vol.3378 of Lecture Notes in Computer Science, pages 283-302, 2005; and Goethals et al., “On private scalar product computation for privacy-preserving data mining,” C. Park and S. Chee, editors, Intl, Conference on Information Security and Cryptology, Vol.2506 of Lecture Notes in Computer Science, pp.104-120, 2004, all incorporated herein by reference.

The basic idea in the protocols of Du et al. is to express the vector x as a sum of M random vectors, i.e., $\sum\limits_{m = 1}^{M}{u_{m}.}$ For every m, Alice sends u_(m) concealed in a set of k random vectors. Bob computes the dot product with y for all the vectors, and adds a random number r_(m) to all products. The results are sent to Alice. Alice and Bob repeat this M times. Finally, Alice and Bob have $\begin{matrix} {\sum\limits_{m = 1}^{M}\left( {{u_{m}^{t}y} + r_{m}} \right)} & {and} & {{- {\sum\limits_{m = 1}^{M}r_{m}}},} \end{matrix}$ respectively, which when added gives the required result.

Du et al. use random permutations. Let π(x) be a vector whose elements are a random permutation of elements of x. The product π(x)^(t)π(y) is equal to x^(t)y. Bob expresses y as the sum of M random vectors, i.e., $y = {\sum\limits_{m = 1}^{M}{v_{m}.}}$ Bob generates m random vectors r_(i) and m random permutations π_(m). For each m, Alice learns π_(m)(x+r_(m)) without learning π_(m) or r_(m). Using a homomorphic encryption scheme as in Du et al., Bob sends π_(m)(v_(m)) to Alice, and Alice computes π_(m)(x+r_(m))^(t)π_(m)(v_(m)). Finally, Alice has ${\sum\limits_{m = 1}^{M}\left( {{x^{t}v_{m}} + {r_{m}^{t}v_{m}}} \right)},$ and Bob has $- {\sum\limits_{m = 1}^{M}\left( {r_{m}^{t}v_{m}} \right)}$ which together form the result. Alice's chances of successfully guessing all elements in y is $\left( \frac{1}{d!} \right)^{M}.$

Another protocol assumes that the number of elements d in the data vector x is even. A vector x₁ is defined as a d/2 dimension vector including the first d/2 elements of the vector x, and the vector x₂ includes the last d/2 elements of x, and x^(t)y =x^(t) ₁y₁+x^(t) ₂y₂. Alice and Bob jointly generate a random invertible d×d matrix M. Alice computes x′=x^(t)M, which is partitioned into x′₁ and x′₂, and sends x′₂ to Bob. Bob computes y′=M⁻¹y, splits it as y′₁ and y′₂ and sends y′₁ to Alice. Alice computes x′₁y′₁ and Bob computes x′₂y′₂ so that their sum is the required result.

Other algebraic protocols for computing dot products are also known, see Ioannidis et al., “A secure protocol for computing dot-products in clustered and distributed environments,” Proceedings of the Intl. Conf. on Parallel Processing, 2002; Vaidya et al., “Privacy preserving association rule mining in vertically partitioned data,” Proceedings of the Intl. Conf. on Knowledge Discovery and Data Mining, 2002; and Ravikumar et al., “A secure protocol for computing string distance metrics,” Proceedings of the Workshop on Privacy and Security Aspects of Data Mining, pp. 40-46, 2004, all incorporated herein by reference.

Classifiers

Data classification is well known. If the data have a multivariate distribution, then the classifier typically uses multivariate Gaussian distributions. Each class can be modeled by either a single multivariate Gaussian distribution or a mixture of multivariate Gaussian distributions.

The classifier determines a value of a discriminant function g _(i)(x)=ln p(x|ω _(i))+ln P(ω_(i)) for all classes ω_(i) and assign the data x to class ω_(i) if g_(i)(x)>g_(j)(x) for all j≠i. Here, p(x|ω_(i)) is a class-conditional probability density function, and P(ω_(i)) is an a priori probability of class ω_(i).

Single Multivariate Gaussian Distribution

The mean vector and covariance matrix of the i^(th) class are μ_(i) and Σ_(i), respectively. Hence, p(x|ω_(i))˜N(μ_(i), Σ_(i)), and the above equation can be expressed as: $\begin{matrix} {{g_{i}(x)} = {{{- \frac{1}{2}}\left( {x - \mu_{i}} \right)^{t}{\Sigma_{i}^{- 1}\left( {x - \mu_{i}} \right)}} - {\frac{d}{2}\quad\ln\quad 2\quad\pi} - {\frac{1}{2}\ln\quad{\Sigma_{i}}} + {\ln\quad{{P\left( \omega_{i} \right)}.}}}} & (1) \end{matrix}$

If the term (d/2) In 2π term is ignored, then equation (1) can be rewritten as: $\begin{matrix} {{{{g_{i}(x)} = {{x^{t}{\overset{\_}{W}}_{i}x} + {{\overset{\_}{w}}_{i}^{t}x} + w_{i0}}},{{where}\quad\begin{matrix} {{{\overset{\_}{W}}_{i} = {{- \frac{1}{2}}\Sigma_{i}^{- 1}}},} & {{{\overset{\_}{w}}_{i} = {\Sigma_{i}^{- 1}\mu_{i}}},} \end{matrix}}}{{{and}\quad w_{i0}} = {{{- \frac{1}{2}}\mu_{i}^{t}\quad\Sigma_{i}^{- 1}\mu_{i}} - {\frac{1}{2}\ln\quad{\Sigma_{i}}} + {\ln\quad{{P\left( \omega_{i} \right)}.}}}}} & (2) \end{matrix}$

Vectors of (d+1)-dimensions, x and w_(i), can be generated by appending the value 1 to the vector x, and appending an element w_(i0) to the vector w _(i). By changing the matrix W _(i) into a (d+1)×(d+1) matrix W_(i), where the first d components of the last row are zeros and the last column is equal to w_(i) ^(t), equation (2) can be written as: g _(i)(x)= x ¹ W _(i) x.

Expressing x as x, the equation becomes: g _(i)(x)=x^(t) W _(i) x.  (3)

Mixture of Distributions Gaussians

The mean vector and covariance matrix of the j^(th) Gaussian distribution in class ω_(i) are Σ_(ij) and Σ_(ij), respectively. Hence, ${{p\left( {x❘\omega_{i}} \right)} = {\sum\limits_{j = 1}^{J_{i}}{\alpha_{ij}{\aleph\left( {\mu_{ij},\sum\limits_{ij}}\quad \right)}}}},$ where J_(i) is the number of Gaussian distributions describing class ω_(i), and α_(ij) are the coefficients of the mixture of Gaussian distributions.

A log likelihood for the j^(th) Gaussian distribution in the i_(th) class is given by $\begin{matrix} {{{l_{ij}(x)} = {{x^{t}{\overset{\_}{W}}_{ij}x} + {{\overset{\_}{w}}_{ij}^{t}x} + w_{ij}}},{{{where}\quad{\overset{\_}{W}}_{ij}} = {{- \frac{1}{2}}\sum\limits_{ij}^{- 1}}},{{\overset{\_}{w}}_{ij} = {\sum\limits_{ij}^{- 1}\mu_{ij}}},{{{and}\quad w_{ij}} = {{{- \frac{1}{2}}\mu_{ij}^{t}} - {\frac{1}{2}\ln{{\sum\limits_{ij}\quad }.}}}}} & (4) \end{matrix}$

Expressing x as a (d+1)-dimensional vector and W _(ij), w _(ij), w_(ij) together as the (d+1)×(d+1) matrix W_(ij), as described above, equation (4) becomes: l _(ij)(x)=x ^(t) W _(ij)x.  (5)

Hence, the discriminant function for the i^(th) class can be written as $\begin{matrix} {{{{g_{i}(x)} = {{\log\quad{{sum}\left( {{{\ln\quad\alpha_{i\quad 1}} + {l_{i\quad 1}(x)}},\ldots\quad,{{\ln\quad\alpha_{{iJ}_{i}}} + {l_{{iJ}_{i}}(x)}}} \right)}} + {\ln\quad{{P\left( \omega_{i} \right)}.{where}}}}}{\log\quad{{sum}\left( {x_{1},\ldots\quad,x_{J_{i}}} \right)}} = {{\max\left( {x_{1},\ldots\quad,x_{J_{i}}} \right)} + {\ln\left( {\sum\limits_{j = 1}^{J_{i}}{\mathbb{e}}^{\Delta_{j}}} \right)}}},{{and}{\Delta_{j} = {x_{j} - {{\max\left( {x_{1},\ldots\quad,x_{J_{i}}} \right)}\quad{\forall_{j}{\in {\left\{ {1,\ldots\quad,J_{i}} \right\}.}}}}}}}} & (6) \end{matrix}$

Hereinafter, the data vector x denotes a (d+1)-dimensional vector with a last component equal to 1, unless stated otherwise.

SUMMARY OF THE INVENTION

The invention provides a method for securely classifying data. A first party has a data vector x to be classified. A second party has a classifier defined as a set of multivariate Gaussian distributions. There is one multivariate Gaussian distribution for each possible class.

A secure inner dot product procedure is applied to each multivariate Gaussian distribution and the data vector x to produce a vector a_(i) for the first party and a vector b_(i) for the second party for each application. The secure inner dot product is then applied to each vector b_(i) and the data vector x to produce a scalar r_(i) for the first party and a scalar q_(i) for the second party for each application.

A summed vector of elements [(a₁x+q₁+r₁) . . . (a_(N)x+q_(N)+r_(N))] is formed, and an index I of the summed vector for a particular element having a maximum value is the class of the data vector x.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a secure method for classifying data with multivariate Gaussian distributions according to an embodiment of the invention;

FIG. 2 is a block diagram of a secure method for classifying data with mixtures of multivariate Gaussian distributions according to an embodiment of the invention; and

FIG. 3 is a block diagram of a secure method for classifying data with mixtures of multivariate Gaussian distributions according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 shows a secure method 100 for classifying data according to one embodiment of our invention. Alice, a client 110, has data in the form of a vector x 111. Bob, a server 120, has a classifier, W_(i) 121, that uses multivariate Gaussian distributions. In this embodiment of our invention shown in FIG. 1, there is one multivariate Gaussian distribution for each data class. In another embodiment shown in FIG. 2, the classifier is in terms of a mixture of multivariate Gaussian distributions for each class.

Single Multivariate Gaussian Distribution

The classifier 121 can be in the form of a matrix W_(i) for i=1, 2, . . . , N. We express the matrix W_(i) as └w_(i) ¹w_(i) ² . . . w_(i) ^(d+1)┘, where w_(i) ^(j) is the j^(th) column of the matrix W_(i).

One embodiment of the invention uses a secure inner dot product (SIP) procedure 130. It should be noted that the steps of the SIP are partially performed by Alice 110 and partially by Bob 120, as described above. The SIP takes as input the vector x 111 and the Gaussian distributions W_(i) 121. A pair of scalar values a_(j) and b_(j) is produced for each application of the SIP procedure 130 to every column w_(i) ^(j) of the classifier W_(i) 121. For every i, Alice obtains all the scalar values a_(j) as a vector a_(i)=[a₁ . . . a_(d+1)] 112 and determines a_(i)x 105, and Bob obtains all the scalar values b_(j) as a vector b_(i)=[b₁ . . . b_(d+1)] 122.

Then, the SIP 130 is applied again for the data vector x 111 and the vector b_(i) 122. This produces the scalar values q_(i) 113 and r_(i) 123, respectively.

Alice construct a vector A=[(a₁x+q₁) . . . (a_(N)x+q_(N))] 114 using a summation 115, and Bob constructs a vector B=[r₁ . . . r_(N)] 124.

Then, Alice and Bob perform a secure maximum protocol (SMAX) 140 on the vectors A and B, and Alice obtains SMAX(A, B), which is the classification I 101 of the data vector x 111.

Correctness: Vectors a_(i) and b_(i) are such that a_(i)+b_(i)=x^(t)W_(i). Also, b_(i)x=q_(i)+r_(i). Hence, x^(t)W_(i)x is given by a_(i)x+q_(i)+r_(i). The value I is equal to the value of index i for which x^(t)W_(i)x is maximum.

Efficiency: For a given i=I, the above protocol has (d+2) SIP calls. Hence, the protocol requires N(d+2) calls to SIP procedure 130 and one call to SMAX.

Security: Neither Alice nor Bob learn the complete result of any dot product. Hence, if the protocols for SIP and SMAX are secure, then the entire protocol is secure.

Mixture of Multivariate Gaussian Distributions

FIG. 2 shows a method 200 for classifying data with mixtures of Gaussian distributions. In this method, there are three calls to the SIP procedure 130, and two calls to the SMAX procedure 140.

As before, Alice 210 has the data vector x 111. Bob 220 now has a classifier 221. In the embodiment shown in FIG. 2, the classifier 221 includes, for each class, a mixture of Gaussian distributions w_(ij), mixture weights au and prior probabilities P(ω_(i)), for i=1, 2, . . . , N and j=1, 2, . . . , j_(i). As output, Alice obtains the class I 101 such that g₁(x)>g_(j)(x) for all j≈I.

The method 200 performs the following steps: The SIP procedure 130 is applied to the vector 111 and to each Gaussian distribution 221 W_(ij) in the i^(th) mixture to obtain intermediate classification indices K_(i) 201, then the SMAX procedure is applied to each pair A_(i) 214 and B_(i) 224, which is derived as described above.

It should be noted that the vector 214 is A_(i)=[A_(i1) . . . A_(iJ) _(i) ] and the vector 224 is B_(i)=[B_(i1) . . . B_(iJ) _(i) ].

In intermediate steps by Bob, a vector B′_(i)=[B′_(i1) . . . B′_(iJ) _(i) ] is constructed from scalar value r_(i) 123, and B_(ij)=B′_(ij)+ln α_(ij) so that A_(ij)+B′_(ij)=l_(ij)(x), where l is the log likelihood as used in equations (5) and (6) above.

Alice and Bob use the indices K_(i) 201 to select elements A_(iK) _(i) 203 and B_(iK) _(i) 204 respectively, from matrices A_(i) 214 and B_(i) 224. Alice constructs a vector U 205 from all elements A_(iK) _(i) .

Alice and Bob determine intermediate vectors A _(i)=[(A _(i1) −A _(iK) _(i) ) . . . (A _(iJ) _(i) −A _(iK) _(i) )], and B _(i)=[(B _(i1) −B _(iK) _(i) ) . . . (B _(iJ) _(i) −B _(iK) _(i) )], respectively, and use these vectors as exponents to form vectors e ^(A) ^(i) 251 and e ^(B) ^(i) 252.

The SIP procedure 130 is applied to the vectors e ^(A) ^(i) 251 and e ^(B) ^(i) 252. The result is φ_(i) 255.

Bob constructs a vector 206, V=[v₁ . . . v_(N)], where v_(i)=B_(iK) _(i) +ln φ_(i)+ln P(ω_(i)). Alice constructs the vector 205, U=[u₁ . . . u_(N)], where u_(i)=A_(iK) _(i) , as described above. The SMAX procedure 140 is applied to the vectors U 205 and V 206, and Alice obtains the class I 101.

Correctness: According to the above protocol, u_(i)+v_(i) is=g_(i)(x), see equation (7).

Efficiency: For a given i, there are (J_(i)(d+2)+1) calls to the SIP procedure 130, and two calls to the SMAX procedure 140.

Security: Alice and Bob do obtain the index of the maximum value in the vector A_(i)+B_(i). It might appear that this leaks some information to Bob about the vector x. Indeed, if Bob knew the index of the maximum of the vector A_(i)+B′_(i), then Bob would know in which Gaussian of the mixture x lies. Bob could use this information from every class to learn about the distribution of the data vector x. However, Bob learns the index only after adding ln α_(ij) to every element. Consequently, the index that Bob learns could be different from the index of the maximum of A_(i)+B′_(i). To eliminate the possibility of Bob learning the index, the following alternative protocol is provided.

Mixture of Multivariate Gaussian Distributions with Secure Maximum Value

FIG. 3 shows a method 300 for classifying the data vector with the mixtures of Gaussian distributions. In this method, there are three calls to the SIP procedure 130, one call to the SMAX procedure 140, and one call to a SVAL procedure 310, described below.

This protocol eliminates the possibility of Bob learning the index of the maximum value as determined above. This protocol is similar to the protocol of FIG. 2.

In this protocol, Alice and Bob use intermediate values y_(i) 311 and z_(i) 312, instead of A_(iK) _(i) 203 and B_(iK) _(i) 204 directly. Thus, the protocol 300 can be expressed as follows:

Input: As before, Alice has the data vector x 111, and Bob has the classifier W_(ij) 221, mixture weights α_(ij), and prior probabilities P(ω_(i)), for i=1, 2, . . . , N and j=1, 2, . . . , J_(i).

Output: Alice learns I 101, such that g₁(x)>g_(j)(x), for all j≈I. Bob learns nothing about the data vector x.

1. For i=1, 2, . . . , N

(a) Alice and Bob perform the steps as described above for J_(i) Gaussians in the i^(th) mixture to obtain the vectors A_(i)=[A_(i1) . . . A_(iJ) _(i) ] and B′_(i)=[B′_(i1) . . . B′_(iJ) _(i) ]. Notice that A_(ij)+B′_(ij)=l_(ij)(x).

(b) Bob constructs the vector B_(i)=[B_(i1), . . . , B_(iJ) _(i) ], where B_(ij)=B′_(ij)+ln α_(ij).

(c) Alice and Bob engage in the secure maximum value SVAL protocol 310 with vectors A_(i) 214 and B_(i) 224 to obtain the intermediate y_(i) 311 and z_(i) 312, i.e., y_(i)+z_(i)=SVAL(A_(i), B_(i)).

(d) Alice and Bob determine respectively, A _(i)=[(A_(i1)−y_(i)), . . . , (A_(iJi)−y_(i))] and B _(i)=[(B_(i1)−z_(i)), . . . , (B_(iJi)−z_(i))].

(e) Alice and Bob determine the dot product between the vectors e ^(A) ^(i) and e ^(Bi) using SIP 130, and Bob obtains the result φ_(i) 255 of the dot product.

2. Bob determines the vector V=[v₁, . . . , v_(N)], where v_(i)=z_(i)+lnφ_(i)+lnP(ω_(i)). Alice determines the vector U=[u₁, . . . , u_(N)], where u_(i)=y_(i).

3. Alice and Bob perform the secure maximum index protocol 140 between vectors U 205 and V 206, and Alice obtains the class I 101.

Correctness: According to the above protocol, u_(i)+v_(i)=g_(i)(x), see equation (7).

Efficiency: For a given i, there are (J_(i)(d+2)+1) SIP calls, and one SVAL call and one SMAX call.

Security: If SIP, SVAL and SMAX are secure, then the above protocol is secure.

Secure Maximum Index (SMAX)

Generally, the SMAX procedure 140 determines an index to an element in a d-element vector [(x ₁ +y ₁), . . . , (x _(d) +y _(d))]=x+y, where the vector x=[x₁, . . . , x_(d)], and the vector y=[y₁, . . . , y_(d)]. For example, if the fifth element is the element with the largest value, i.e., (x₅ +y₅), then the index 5 is produced. A similar procedure can be used to determine a minimum index.

Our procedure is a novel extension of a procedure described by Du et al., “Privacy-preserving cooperative stanalysis,” Proceedings of the 17th Annual Computer Security, December 2001.

Bob generates a random polynomial in two variables f(x, y)=f(x+y), such that f(z₁)≦f(z_(j)) if and only if z_(i)≦z_(j). For each i=1, 2, . . . , d, Alice uses oblivious polynomial evaluation (OPE) to determine h_(i)(x_(i)), where h_(i)(x)=f(x, y_(i)). The index for which h_(i)(x_(i)) is the maximum is the answer that Alice desires. Notice that neither party can determine the value of the maximum element, but Alice can determine the order of elements in the vector x+y.

Secure Maximum Value Protocol (SVAL)

Alice has a vector x=[x₁, . . . , x_(d)], and Bob has the vector y=[y₁, . . . , y_(d)]. Alice and Bob would like to determine the value of the maximum element in z=x+y. After the protocol, neither should party know the index of the maximum element. Notice that the same protocol can be used to compute the value of the minimum value.

First, consider a naive approach. Notice that z_(i)≧z_(j)

(x_(i)−x_(j))≧(y_(j)−y_(i)). Alice and Bob can do such pairwise comparisons and mimic any conventional maximum finding process to learn the value of the maximum. To perform the comparisons securely, Alice and Bob can use Yao's millionaire protocol, see Yao, “Protocols for secure computation,” Proc. of the 23rd IEEE Symposium on Foundations of Computer Science, pp. 160-164, 1982.

However, if Alice and Bob follow that naive approach, then both will be able to also find the index of the maximum. In an improvement, Alice and Bob obtain two vectors whose sum is a random permutation of z. Neither Alice nor Bob know the permutation. Then, Alice and Bob can follow the above naive approach on their newly obtained vectors to determine additive shares of the maximum element.

Permute Protocol

Input: Alice and Bob have d-component vectors x and y, respectively. Bob has a random permutation π.

Output: Alice and Bob obtain q and s such that q+s=π(x)+π(y).

-   -   1. Alice generates public and private keys for homomorphic         encryption and sends the public key to Bob. Let E(·) denote         encryption with Alice's public key.     -   2. Alice encrypts each element of the vector x and sends the         resulting vector x to Bob.     -   3. Bob generates a random vector r and computes a new vector θ,         where θ_(i)= x _(i)E(r_(i))=E(x_(i)+r_(i)), for i=1, . . . , d.     -   4. Bob permutes θ and sends π(θ) to Alice. Alice decrypts the         vector to obtain q.     -   5. Bob computes y−r, and then permutes it using r to obtain s         π(y−r).

Alice and Bob engage in the above permute protocol twice, the second time with their roles interchanged. After this is done, Alice and Bob will have two vectors whose sum will be a random permutation of the original sum but neither of them will know what the permutation is. They can then follow the naive maximum finding approach to compute additive shares of the maximum element.

Although the invention has been described by the way of examples of preferred embodiments, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention. 

1. A computer implemented method for classifying data, comprising the steps of: generating, by a first party, a data vector x to be classified; defining, by a second party, a classifier with a set of multivariate Gaussian distributions, there being one multivariate Gaussian distribution for each possible class; applying, to each multivariate Gaussian distribution and the data vector x, inner dot product procedures to produce a vector a_(i) for the first party and a vector b_(i) for the second party for each application; applying, to each vector b_(i) and the data vector x, the inner dot product procedure to produce a scalar r_(i) for the first party and a scalar q_(i) for the second party; forming a summed vector of elements [(a₁x+q₁+r₁), . . . , (a_(N)x+q_(N)+r_(N))]; and determining an index of the summed vector for a particular element having a maximum value, the index being a class of the data vector x.
 2. The method of claim 1, in which the inner dot product procedure is a secure cryptographic protocol.
 3. The method of claim 1, in which the inner dot product procedure is a secure algebraic protocol.
 4. The method of claim 1, in which the classifier is a matrix W_(i) for i=1, 2, . . . , N, for N classes, and the matrix W_(i) that describes the parameters of the Gaussian distribution for class i is └w_(i) ¹w_(i) ² . . . w_(i) ^(d+1)┘, where w_(i) ^(j) is the j^(th) column of the matrix W_(i).
 5. The method of claim 1, in which the index is determined securely.
 6. The method of claim 1, in which there is a mixture of multivariate Gaussian distributions for each possible class i, and in which each multivariate Gaussian distribution W_(ij) is associated with mixture weights α_(ij) and prior probabilities P(ω_(i)), for i=1, 2, . . . , N and j=1, 2, . . . , J_(i), where J_(i) is the number of Gaussian distributions in the i^(th) mixture.
 7. A computer implemented method for classifying a vector x generated by a first party with a classifier provided by a second party, in which the classifier is defined as a set of multivariate Gaussian distributions, there being one multivariate Gaussian distribution for each of N classes, and in which the classifier is expressed as a matrix W_(i) for i=1, 2, . . . , N, and the matrix W_(i) is expressed as └w_(i) ¹w_(i) ² . . . w_(i) ^(d+1)┘, where w_(i) ^(j) is a j^(th) column of the matrix W_(i), comprising the steps of: (1) performing a secure inner product procedure SIP (x, W_(i) ^(j)) to obtain vectors a_(i)=└a_(i) ¹ . . . a_(i) ^(d+1)┘ and vectors a_(i)x for the first party and a vector b_(i)=└b_(i) ¹ . . . b_(i) ^(d+1)┘ for the second party, for i=1, 2, . . . , N and then for j=1, . . . , d+1, and performing the secure inner product procedure SIP (b_(i), x) to obtain a scalar q_(i) for the first party and a scalar r_(i) for the second party; (2) constructing a vector A=[(a₁x+q₁) . . . (a_(N)x+q_(N))] by the first party and a vector B=[r₁ . . . r_(N)] by the second party; and (3) performing a secure maximum index protocol SMAX (A, B) to obtain a class I associated with the data vector x by the first party.
 8. The method of claim 7, in which the classifier is defined as a matrix W_(ij), mixture weighs α_(ij) and prior probabilities P(ω_(i)), for i=1, 2, . . . , N and j=1, 2, . . . , J_(i), and further comprising the steps of: (1) for i=1, 2, . . . , N (a) performing the steps (1) and (2) of claim 7 for the J_(i) multivariate Gaussians distributions to obtain vectors A_(i)=[A_(i1) . . . A_(iJ) _(i) ] for the first party and vectors B′_(i)=[B′_(i1) . . . B′_(iJ) _(i) ] for the second party such that A_(ij)+B′_(ij)=l_(ij)(x); (b) constructing a vector B_(i)=[B_(i1) . . . B_(iJ) _(i) ] by the second party, where B_(ij)=B′_(ij)+ln α_(ij). (c) performing a secure maximum index protocol SMAX (A_(i), B_(i)) to obtain indices K_(i) by the first party and the second party; (d) constructing vectors A _(i)=[(A_(i1)−A_(iK) _(i) ) . . . (A_(iJ) _(i) −A_(iK) _(i) )] by the first party and vectors B _(i)=[(B_(i1)−B_(iK) _(i) ) . . . (B_(iJ) _(i) −B_(iK) _(i) )] by the second party; and (e) performing the secure inner product SIP (e ^(A) ^(i) , e ^(B) ^(i) ) to obtain φ_(i) for the second party; (2) constructing the vector u=[u₁ . . . u_(N)] by the second party, where u_(i)=B_(iK) _(i) +lnφ_(i)+lnP(ω_(i)), and constructing the vector v=[v₁ . . . v_(N)] by the first party, where v_(i)=A_(iK) _(i) ; and (3) performing the secure maximum index protocol SMAX (u, v) to obtain the class I associated with the data vector x by the first party. 